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ABSTRACT 


A new discrete reliability growth model is created by 
modifying the often used Army Material Systems Analysis 
Activity (AMSAA) continuous reliability growth model. The new 
model is labeled the AMSAA-D model. Its accuracy is evaluated 
and compared with three other existing discrete reliability 
growth models. The results show the AMSAA=-D model is at least 
as accurate as the other models. In particular it is more 
accurate than an AMSAA discrete model which requires computer 
supported numerical methods to calculate the reliability 
estimates from test data. The AMSAA-D reliability estimates 
can be made with a hand-held calculator. Computer simulations 
were used to generate test data needed for the evaluation. The 
Simulated test plan assumes that repeated tests on a system 
are performed until a predetermined number of failures occur, 
at which time a design change is made to the system to improve 
its reliability. Ten reliability values are specified to 
define a reliability growth pattern. Five hundred replications 
of each growth pattern are simulated. For each replication, 
reliability estimates are calculated for each of the ten sets 
of generated test data using equations from each of the four 


growth models. Averages and sample mean square error values 


lv 


across the 500 replications are used to determine accuracy. 
Sensitivity of the AMSAA-D model to the number of failures 
before system modification and to the number of possible 
failure causes is also evaluated. Results of all evaluations 


are presented graphically. 


THESIS DISCLAIMER 


The reader is cautioned that computer programs developed 
in this research may not have been exercised for all cases of 
interest. While every effort has been made, within the time 
available, to ensure that the programs are free of 
computational and logic errors, they cannot be considered 
validated. Any application of these programs without 


additional verification is at the risk¥oe thesucon 
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I. INTRODUCTION 


Early in the acquisition process of a system, a variety 
of test programs are conducted to detect weaknesses in its 
design features and manufacturing processes. These tests are 
performed under tight budget constraints; consequently the 
number of component, assembly and system tests is severely 
limited. These development testing programs are designed to 
induce early failures and determine associated failure causes. 
This information is needed to make appropriate changes in the 
design and production processes. Hopefully these changes will 
yield an iGeease in the reliability of the system under 
acquisition. Current DoD policy requires reliability growth 
management and assessment programs for major systems 
acquisitions. 

Reliability growth methodology used to estimate the actual 
reliability growth patterns should be able to track a variety 
of growth patterns reasonably well. Many reliability growth 
models have been developed and reported in the literature. 
New ones are continually being developed and proposed for use 
my contractors. 

A discrete version of the well known Army Material System 


Analysis Activity ( AMSAA ) or DUANE model is analyzed in this 


thesis. This discrete model uses the number of observed tests 
to achieve a prescribed number of failures as the primary data 
input. This model is evaluated and compared with other 
established models which have been extensively tested and 
described in former theses [Ref. 1,2,3]. The same Monte Carlo 
Simulation is used as in the former theses to make the 
comparisons. The simulation was originally developed by James 
E. Drake [Ref. 1], then twice updated and enlarged in its 
performance and ability to track different reliability growth 
patterns by James D. Chandler, Jr. [Ref. 2] and Pam Markiewicz 
[Ref. 3]. The program and its features are well described in 
these theses and will not be included here. Only the program 
introduced to estimate the new model is presented. 

In Chapter II the analysis of the proposed model is 
developed, the simulation data presented and the results 
evaluated. A summary and recommendations for further studies 


is presented in Chapter III. 


II. RELIABILITY GROWTH MODELING 


A proposed reliability growth model should be able to 
track accurately all likely reliability growth patterns. 
During the development phase of a system, the actual pattern 
of reliability growth is dependent on many _ = factors. 
Reliability progress paths may be steadily increasing or they 
may indicate early stagnation and degradation prior to 
continued growth. They may also demonstrate no reliability 
progress at all [Ref. 4]. Figure 1 shows the true reliability 
growth patterns which will be used for evaluation purposes in 
this thesis. The different patterns are numbered from 1 to 6. 
Pattern numbers cited later in this thesis refer to Figure 1. 

Reliability growth usually occurs in discrete jumps; 
therefore it is meaningful to break the system tests into 
subphases. Tests within the same phase have the same 
probability of success, Di piCers the same reliability. 
Reliability may change from phase to phase. Hopefully it is 
increasing as the items under test undergo change from phase 


to phase. 
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Figure 1 : Possible reliability growth patterns 


A. THE DISCRETE AMSAA MODEL 
1. Theory 
The Discrete AMSAA model was developed in 1983 by L. 
Crow [Ref. 5] It stems from a cumulative continuous model of 
the form 
K(t) = at? 
where K(t) denotes total number of failures after t hours of 


testing. 


In the discrete model 


i 
Kou M. 
i j=1 J 
where M. denotes the number of failures in test phase j. 
a 
= = N. 
i Pig 2 


where Ns denotes the number of tests in test phase j. 


The model then states 


econ 2(M,)) - (2 = R)N, = are 


a — ie s Be 
E(K,) = E(M, - M.) = AT, + (1 R,)N, = ATS 
2 Se ee iP 
tmaas (1 R,)N, = AT. AT) 


Continuing in this manner the AMSAA discrete model states 


(1 - RN, = Te - xPe_, 
Maximum likelihood estimates for X and 8B are used to 
obtain the MLE for R,. The estimates are updated sequentially 
as the test results of the next test phase become available. 
2. Some Results 
The results shown in the following figures are taken 
directly from a thesis by Major Rio M.Thalieb [{Ref. 6]. 
Figure 2 shows the estimation of the reliability 
growth when the actual reliability is following pattern 1. 
Figures 16 and 17 in Appendix D show the estimation 


of the discrete AMSAA model for patterns 2 and 4. 


RELIABILITY GROWTH ESTIMATION 
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Figure 2 : AMSAA reliability growth estimation, pattern 1 


It is clear that the discrete AMSAA model is not able 
to track all possible reliability growth patterns, especially 
when the growth is not steadily concave increasing. This is 
an expected result due to the fact that the results of all 
phases are combined to estimate the reliability. 

B. THE AMSAA-D MODEL 

The model analyzed in this thesis is a discrete variation 
of the continuous AMSAA model for estimating the failure rate 
of a system. The input random variables to the continuous 


model are the total number of failures and the total 


accumulated test time up to the point of estimation [Ref. 4]. 
It is different from the discrete AMSAA model developed by 
Duane and evaluated in the thesis of R.M. Thalieb.[Ref. 6]. 
The equation used by AMSAA to model the instantaneous failure 


rate fon fe 


fop = b(1 - a)TT? (ae. ) 


where TT denotes the total test time and a and b are 
parameters. The concept for this model was first presented by 
Duane [Ref. 7]. A large amount of development test data he 
examined indicated that the logarithm of the failure rate was 
a linear function of the logarithm of the total accumulated 
test time. Comments on the development of this model appear 
in Appendix E. Woods [Ref. 4] showed that a modification to 
this model, using regression estimation methods rather than 
maximum likelihood estimation methods, provided a more 
accurate growth model. The new discrete model developed here 
uses Similar regression estimation methods. 
1. Theoretical analysis 
The discrete modification of the continuous AMSAA 
model developed here should be.distinguished from the Discrete 
AMSAA model and will hereafter be referred to as the AMSAA-D 
model. We shall adopt the following notation 
J or k : test phase index, corresponds to design change 


index 


fs >: actual failure rate during phase j. Equal to 


probability of failure for a test. 


f+, : failure rate at the end of phase k as determined by 
the model. 
Fj: total number of failures observed in phase j 
Nj: number of trials in phase j 


Ny; : total accumulated number of trials up) Commas 
including phase k 
( Ne, = Ny + No + ..- + Ny ) 
Cy number of possible failure causes in 
phase j 
We shall model the failure rate f,, after agjeeea 


number of tests Ny, have been accumulated over k phases =by 
-a 
f+}, = Jeet ie a)Nep, ( Live ) 


This provides a discrete version of the model given 
by equation (II.1). Regression methods similar to those used 
by Woods [Ref. 4], will be used to estimate the parameters a 
and b chronologically as test phases are completed. That is, 
all observed numbers of failures Fy, F5, -:.- Fy) andieeme 
observed total numbers of trials N,, Noy, ... Ny, are used to 
obtain the current estimate of fi. k= loa 

Fach test phase is terminated after a predetermined 
number of failures has occurred. Consequently the number of 


failures per phase will always be greater than 0. The maximum 


likelihood estimate for f. is F/N;. Its expected value is 
larger than f., the probability of failure on each trial. 
There are numerous ways to decrease the bias. The reduction 
factor chosen here is 2€5/(142C.) primarily because it 


resembles the bias reduction factor used by Woods [Ref. 4]. 


Therefore a more nearly unbiased estimate f. or f5 is used 


where 
PALES je 
5s 
t a (II.3) 
= 
O60 a= bie. = 1; 2 
Ns d) 
A rationale for equation (II.3) can be developed from 
a continuous analog. If X; = min(T;,to), io 12a... ,N where 
aT T,, are iid exponential variables, then a nearly 


> td Ie RLY N 


unbiased estimate \ for the common failure rate \ is given by 


2N Fe 
ae 2X 5 


where F is the number of failures (Tt; < 1 This bias 


0): 
correction factor, 2N/(1+2N) is developed in Reference 8. 
Woods [Ref. 4] uses this correction factor for the continuous 
model. It is modified here by replacing Ns with Cs. Table I 
shows a comparison of both correction factors 2C5/(1+2C,) and 


pm (ten. ). The chosen correction factor in equation (II.3) 


is a constant when the number of failure causes or the number 


of subsystems in each phase are fixed. Table I shows both the 
smallest and the largest values of the correction factor 
2N./(1+2N.) across all 500 replications and six growth 
patterns for j = 1 andj} = 10, i.e. for phases 1 and 10. These 
four values are grouped by number of failure causes. The 
number of failure causes is an input parameter for the 
simulation program. The number of failure causes is important 
when a fraction of a failure for a specific cause can be 
removed after a fixed number of successfull system tests 
occur without failure for the same cause. This failure 
reduction process is called failure discounting and is 
permitted by specifying input parameters to the computer 
program used to run these simulations. Failure discounting was 


not allowed in any simulation in this thesis. 


TABLE I : COMPARISON OF CORRECTION FACTORS 










# of failure ‘causce-ucer 
AMSAA-D 2C./(1+2C.) 0.8571 | 0.9091 Uomeeee 
O 


min} 0.8571 OTS5 72 eave 0O4857m 
phase 1 
max] 0.9524 O02 9565 0.9565 0.9322 
: O 
O 














continuous 
AMSAA 
model 

2N,/(1+2N,) 9524 | 0.9091 | 0.96 923m 


Ae bo 4? 







0 
OE ae NS: O2996 0.9948 









Least square estimation procedures are used to obtain 
estimates for a and b as the testing in each new phase is 


completed. This provides update estimates for a and b and thus 
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for the failure rate f, jo eee encuatlons for the least 


a 
square estimates were first developed by Neal [Ref. 9]. He 


uses the transformations 


X5 = ln Ne5 and 


Ys = ln f+; 


to obtain a linear relationship between Ys and Xs His 


equations for the least squares estimates Ay and 6, are as 


follows 
i = k 
a X.Y. - Y > x 
A j=1 J 3 k j=1 J 
a — 
Kk Peek k 9 
Pea | Ken Ke 
a) j=1 J 
ieee al Bok ate ay xy 
i: 1-4 
a 
wher X, = 2 ye 
ere ky 5 
- Jt 
Ye = 7 2 ¥5 
Keine 


The regression estimation can be made only after two 
phases of testing have been completed. Therefore the failure 


rate is estimated for k > 2 as 


A -a 
fee = (1 - a ) B, NLUX (Gaede) 


The term fey - f. is estimated with equation ( II.3 ). 


itl 


2. Reliability estimate as a linear function of the 
failure rate estimate. 

To compare the results of this model with other 
models, it is necessary to convert the failure rate estimate 
into a réelvabpiilatyewalue, 

A discrete failure rate is defined as 

P([N=n] 


= =e poe : 
fn * Sone ( Tom 


where the random variable N is the number of cycles (trials) 


th failure occume 


up to and including the cycle on which the r 
(Ref. 10]. For the geometric distribution r = 1 and f. = 
When estimating system reliability in each phase, one basic 
assumption is that the unknown reliability, p, stays constant 
within that phase. Even though we are observing the number of 


trials torthe pth 


failure with common probability of success 
p on each trial, we shall take the failure rate to be the 
unreliability i1-p. This means that the failure rate is just 
the probability of failure ona single trial inside one single 
phase. Conseguently one method for converting the least 
squares failure rate estimates into reliability estimates is 


from the expression 


RR See ae ( Irseme 


where f,) is given in equation (I1.4). In order to avoue 


subscripts in graphs, equation (II.6) will be written as 


eZ 


R(N) = 1 - F(N) in the graphs. In the text that follows it is 
referred to as "the linear model". A more detailed discussion 
of the relationship between failure rate and reliability for 
G@iscrete distributions is provided in Appendix C. 
3. Reliability estimate as an exponential function of the 
failure rate estimate. 

Analysis of the first simulations indicated that the 
linear model consistently underestimated the actual system 
reliability on the average. To increase the value of the 
average point estimate, another equation was introduced for 
estimating reliability using the failure rate. This equation 
is 


ene ati (Srr.7 ) 


and is referred to as " the exponential power model " in the 


text. In the figures it is noted as 
R(N) = exp(f(N)). 


4. Simulation results 
The simulations were replicated 500 times for each 
combination of input parameters. These parameters are 


- the phase reliabilities Py Six different sets of 


Pio’ 
these 10 values establish six different reliability growth 


patterns, 


~ the number of failure causes per phase 


Ae, 


-~ the number of failures before corrective action sin eae 


phase. 
Table II gives an overview of all simulated parameter 


CoMmoilnaclonse 


Table II : ALL SIMULATED PARAMETER COMBINATIONS 


# of a (et 
causes fail/phase 





During each replication the estimated system 


reliability as well as the estimation error, Err, was 


computed. Err. is defined as 


Err, = Ry = Re ( IT. 3 
where 
R. >: actual system reliability in phase j 
Re >: estimated system reliability at the end of 
phase j 
These computations are performed for both AMSAA-D 
models ( the linear and the exponential power model ). They 


are also computed for the Maximum Likelihood Estimation model 


and the exponential Regression Estimate model; hereafter 
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referred to as the MLEwd and Exp.Reg.Estimate models. These 
models have been evaluated by Chandler [Ref. 2]. Additional 
comments on these models are given in Appendix E. After all 
replications are completed the average system reliability 
estimate and the Mean Square Error are computed for each 


estimation method as 








ee 1 nrep me 
Ri4= acts — (Rides formal) 7 ( i. 9a} 
, nrep 5 , 
MSE. Se Sea an [EEE | former ~) Get; 9pm) 
where j : phase index 
nrep : total number of replications ( = 500 ) 
Res > average reliability estimate over nrep 
replications for phase j 
MSE >; Mean Square Error 


This average reliability estimate and the Mean 
Square Error are plotted and graphed to show the performance 
of the growth models. 

The MLEwd and the exp.Reg.Estimate models are used to 
compare the performance of both AMSAA-D models relative to 
these former established growth models. The AMSAA-D model does 
not allow the application of any discounting method. Therefore 
the comparisons are all made without any discounting applied 


to the MLEwd and the exp.reg.Estimate. 
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a. Sensitivity of the reliability growth estimation with 

respect to different growth patterns 

The evaluation of the sensitivity of both 
AMSAA-D growth models with respect to different growth 
patterns is discussed with the following parameter setting 

- number of failure causes : 5 
- number of failures/phase : 3 

The results and conclusions, however, are in general the same 
for all other parameter combinations as is shown in Figures 
18 - 35 in Appendix D. 

Figure 3 illustrates the performance of both 
AMSAA-D models ( linear and exponential ) when the true 
reliability growth follows the path stipulated in pattern 1. 

The exponential power model in the first 6 phases 
far overestimates the actual reliability. The linear model 
tracks the convex growth pattern very well over all phases. 
Also it underestimates the actual reliability, which is 
usually a desired behavior. In comparison with the MLEwd and 
the exp.Reg.Estimate, the linear model performs better over 
all phases with respect to the average of the point estimates 
Peale oe 

The Mean Square Error for all four methods is 
shown in Figure 3. The exponential power model is the worst 
estimation model with respect to the average reliability 
estimate, but it has the smallest Mean Square Error and is 
less dependent on the phase. But from phase 8 on, all models 


have nearly the same Mean Square Error. 
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Figure 4 illustrates the performance of all models 
under pattern 2. 

Both AMSAA-D models track the decline of the 
actual reliability very well. The linear model again always 
underestimates the actual system reliability, and the 
exponential power model overestimates the actual system 
reliability up to phase 6. It also smoothes the decline and 
the following incline of the actual reliability. 

The linear model performs better with respect to 
the average reliability estimate, when compared with the MLEwd 
and the exp.Reg.Estimate. Also, from phase 3 on, both AMSAA- 
D models have the smaller Mean Square Error and this Mean 
Square Error is not very strongly influenced by the phase as 
is the case for the MLEwd and the exp.Reg.Estimate. 

For pattern 3 the general behavior of all models, 
as shown in Figure 5, is similar to the behavior for patterns 
lL fancdie2: 

All models perform well under pattern 4 and 5 as 
it can be seen in Figures 6 and 7. For pattern 5 only the 
exp.reg.Estimate shows an overestimation of the actual 
reliability over all phases. Again, the decline of the Mean 
Square Error from phase 1 to phase 3 for all models is very 
Siqgnitpreant. 

Figure 8 shows the behavior under pattern 6. All 
models underestimate the actual reliability quite well in the 
average. The Mean Square Error is nearly the same for all 


models and is also nearly constant over all phases. Only the 
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Figure 4 : Reliability growth estimation, pattern 2 
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Figure 5 : Reliability growth estimation, pattern 3 
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Figure 7 : Reliability growth estimation, pattern 5 
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Figure 8 : Reliability growth estimation, pattern 6 


rae 


exp.Reg.Estimate shows a slight decline in the Mean Square 
Error from phase 4) cemre- 

Figures 9 and 10 illustrate the behavior of both 
AMSAA-D models with respect to the Mean Square Error and with 


phases and patterns as variables. 


MEAN SQUARE ERROR : CAUSES = 6, FAILURES/PHASE = 3 
RCN) ® 1 - FCN) 


@.66 





Figure 9 : Linear model, Mean Square Error as a function of 
phases and patterns 


Both models show a high Mean Square Error in the 
early phases. This 1s reasonable because in the early phases 
less information is available to get accurate reliability 
estimates. The exponential power model ( Figure 10 ) shows a 


constant decline of the Mean Square Error for pattern 1, 
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MEAN SQUARE FRROR : CAUSES * 8, FAILURES/PHASE = 3 


RON) = @exp(f (Nd) 9-7 


Figure 10 : Exponential power model, Mean Square Error as a 
function of phases and patterns 


whereas for the pattern 2 - 5 the Mean Square error drops 
rapidly after phase 1 and stays nearly constant thereafter. 

For both models pattern 6 is less dramatic because 
both models show a nearly constant low Mean Square Error over 
all phases. 

b. Sensitivity of the reliability growth estimation to the 

number of farlure causes. 

To evaluate the sensitivity of the reliability 
growth estimation to the number of failure causes’ the 


Simulation was run for each pattern with the number of failure 
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causes set at 3, 5 and 7. Figure 11 and the Figures 36 to 40 
in Appendix D show three dimensional plots of the Mean Square 
Error as a function of the phase and the number of failure 
causes for the linear and the exponential power model. The 
Mean Square Error was chosen as a measure of the sensitivity, 
because the estimation of the actual reliability doesn't show 
great differences due to the change in the number of failure 
causes. The graphs of the reliability estimation and their 
related Mean Square Error for all different parameter settings 
is enclosed in Appendix D ( Figures 18 - 35 ). 

As can be seen, the general behavior of the growth 
estimation due to the number of failure causes is relatively 
independent of the simulated growth pattern. Also the 
difference between the two models is not very large. Both 
models show, in general, an increasing Mean Square Error when 
the number of failure causes is increased. This increase is 
overall a little steeper for the linear model, but it becomes 
less dependent in the number of causes in later phases. 

The exponential power model shows a more 
differentiated behavior. It has a steep increase of the Mean 
Square Error for pattern 1 in the first 5 phases as the number 
of causes is increased. This is true in all other patterns 
Only for phase 1. For all other patterns and phases greater 
than 1 the Mean Square Error of the exponential power model 
is only slightly increasing with increasing number of causes. 


The slope is reduced even more within the last phases. 


26 


— ee 





MEAN SQUARE EPROR : PATTERN i 
MSE = i - FCN) 
ar 


ie : 





MSE 
@ 





~63 a 
, i 
@.62 > “ 
@.e4 a La 
> a 5 
@ 
* of causes 
3 
MEAN SQUAPE ERROR : PATTERN i 
MSE = exp(f (N)) 
@.06 
@e,es 
8.804 
W 
2 @e.e3 
6.82 
at. 
@.61 





4 & s 
phaves 


Figure Ji: Mean Square Error as a function of phases and 
failure causes, both AMSAA-D models, pattern 1 
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To catch the general trend in a different way a 


total average Mean Square Error was computed as 





1 npat 1 nphase 

COot.avg.MSE = npat _“ ( nphase 2 MSESy ) (II. Doe 

p=1 J=1 
where 
MSE. : é 
Jp : Mean Square Error aS 1n equation (ii. 

5) : phase index ( here 1 to 10 ), 

p : pattern index ( here 1 to 6 ), 

nphase : number of phases ( here 10 ) and 

npat >: number of patterns ( here 6 ). 


The result is shown in Figure 12. It also affirms 
the trend estimation from the three dimensional graphs. The 
total average Mean Square Error increases for both models. 
This is also true for the other two models, MLEwd and 
exp.Regr.Estimate, shown for comparison. But the slope of the 
linear model is steeper than the slope of the exponential 
power model when increasing the number of causes from 3 to 5. 

c. Sensitivity of the reliability growth estimation to the 
number of failures per phase 

The evaluation of the sensitivity of the 
reliability growth estimation to the number of failures per 
phase was done with the following parameter settings 

= all patterns ©) Gomes) 


- number of failure causes = 5 


28 


MEAN SQUARE ERROR ! AVERAGE OF ALL PATTERN AND PHASES 
eerneitivity to rumber of causes 


7 ot xl ee 
-+ REND Bexp CP (ND) 


_ MLE with discounting 


i : 
-p exp,Reg.Eetimate - 


6 
runmber of caeuse@a 


Figure 12 : Average Mean Square Error as a function of failure 
causes 





Figure 13 and Figures 41 to 45 in Appendix D show 
the three dimensional plots of the variation of the Mean 
Square Error over all phases with varied failures/phase for 
all patterns. 

From all Figures, it can clearly be seen that the 
greatest impact on the reduction of the Mean Square Error 
occurs where the number of failures/phase changes from 1 to 
2. In general an increase in the number of failures/phase from 
1 to 2 will reduce the mean Square Error by at least 50% in 
most of the cases. This is relatively independent of the 


phase. But the reduction in phase 1 is always the smallest for 
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MEAN SQUARE ERROR : PATTERN i 





Figure 13 : Mean Square Error as a function of phases and 
failures per phase, both AMSAA-D models, pattern 1 
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each pattern. Note that there is no decrease of the Mean 
Square Error for the exponential power model in the first 
phase for the patterns 1 to 5 when the number of 
failures/phase is increased. Only pattern 6 shows a decrease 
in phase 1 also. 

The variation of the total average Mean Square 
Error over all phases and patterns, computed as in equation 
II.10, with increasing number of failures/phase is shown in 
Figure 14. It again affirms in general the trend evaluated 
from the three dimensional graphs. The greatest reduction of 
the Mean Square error is achieved by increasing the number of 
failures/phase from 1 to 2. This results in an average 
decrease of the Mean Square Error from 0.08 to 0.03 for the 
linear model and from 0.03 to about 0.015 for the exponential 
power model. This result is true in general for the Maximum 
Likelihood Estimation and the exponential Regression Estimate, 
which are also shown in Figures 13 and 14 for comparison. 

d. Comparison with the discrete AMSAA model 

The results show the AMSAA-D model is more 
accurate than the discrete AMSAA model. The AMSAA-D model 
appears to account more accurately for different reliabilities 
in different phases, but still uses information from previous 
phases to yield more accurate estimates of reliability at 
later stages in the growth process. It makes good use of the 
fact that the actual reliability is not constant over all 
phases, but stays constant within a phase. The discrete AMSAA 


model is not able to track all different possible patterns of 
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Figure 14 : Average Mean Square Error as a function of 
failures per phase 


reliability growth as can be seen in Figure 15, which is taken 

from R.Thalieb [Ref. 6] for pattern 2 with 3 failures/phase 
and 1 failure cause. 
e. Conclusion 

The linear AMSAA-D model performs reasonably well 

and is relatively independent of the pattern of actual 

reliability growth. It always underestimates the actual 

reliability and is therefore a conservative approach. It also 

performs at least as good as the formerly established MLEwd 


and exp.Regr.Estimate models when using Mean Square Error as 
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Figure 15 : Comparison of the discrete AMSAA model with the 
AMSAA-D model, pattern 2 


a measure of performance. It is not very sensitive to the 
number of failure causes and this low sensitivity is even 
decreasing within later phases of testing. When the number of 
failures/phase are increased from 1 to 2, the Mean Square 
Error can be reduced by about 50%. 

The exponential power model has a very low Mean 
Square Error. In general, it is about 50% less than the Mean 
Square Error of the other models. It overestimates the actual 


reliability in the early phases for nearly all of the six 
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growth patterns. The impact on accuracy due to an increase in 
the number of failure causes is nearly the same as for the 
linear model. The same can be said for the impact of an 
increase in the number of failures/phase. 

The linear AMSAA-D model is a conservative, but 
reasonable accurate tool, for estimating actual system 
reliability growth. It is not very sensitive to the number of 
failure causes and an increase in the number of failures per 


phase can reduce the Mean Square Error significantly. 
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III. SUMMARY, RECOMMENDATIONS 


A. SUMMARY 

The objective of this thesis was to evaluate the accuracy 
of a discrete modification of the continuous AMSAA model 
( AMSAA-D model ) for the reliability growth estimation. The 
sensitivity of the model to different growth patterns and 
system parameters as the number of subsystems, the number of 
phases and the number of failures till phase termination was 
evaluated. The accuracy of this model was compared with that 
for the MLEwd and the exponential regression estimate models 
[fRef. 1,2,3]. Although possible improvements due to different 
discounting and weighing methods for the MLEWD and the 
exponential regression estimate models have been demonstrated 
in these former theses, comparisons in this thesis were made 
without using discounting or weighting of data. 

Throughout the simulations the AMSAA-D model illustrated 
a reasonable accuracy and was at least as good as the MLEWD 
or the exponential regression estimate. It is able to track 
all possible growth patterns reasonably well. Its sensitivity 
to the number of failure causes and the number of failures 
within a phase compares favorably with that of other 


established models. 
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B. 


RECOMMENDATIONS 


The following are recommendations for further study 


Th 


The AMSAA-D model should be modified by using the 
minimum variance unbiased estimate of the failure rate 
instead of the failure rate estimate using a 
SOrrection. factor. 

Since the exponential regression estimate can be 
improved by weighting the data, similar methods should 


be analyzed for the AMSAA-D model. 
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APPENDIX A 
USER'S GUIDE TO : RELIABILITY GROWTH WITH THE AMSAA-D MODEL 


( DISCONT ) 


iw Lntrogucti.0on 
oo. Ties CONDMayEe Alo rile 
3. The input data file 


4. The output file GROWTH EST Al 


oF 


APPENDIX A : USER'S GUIDE TO 
RELIABILITY GROWTH 

WITH THE 

AMSAA-D MODEL (DISCONT) 


1. Introduction 
In order to use the Fortran program DISCONT the user 
must possess three files 
- Input file "like" PT1C5F3 DATA Al 
- DISCONT FORTRAN Al 
=> DISCON PSE Ee oat 
A sample of each of these files along with a sample 
run using DISCONT EXEC Al and a sample output is contained in 
Appendix B. The input file and the exec file can be changed 
to meet the needs of the user. In the current form, the exec 
file produces a large number of intermediate calculations for 
both the DISCONT FORTRAN program and Lt P. Markiewicz's 
program DRG FORTRAN [Ref. 3]. These calculations may not be 
of interest and therefore may be eliminated with no effect to 
the simulation. A detailed explanation of each file, which is 
created in addition to Lt P.Markiewicz's version is contained 
in the following sections. 
2. The DISCONT EXEC Al file 
The DISCONT EXEC Al contains all necessary file 
definitions and commands needed to run DISCONT FORTRAN Al. 
Appendix B 2. shows a sample run using DISCONT EXEC Al. 


DISCONT FORTRAN Al is an extension of the DRG FORTRAN Al 
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program of Lt P.Markiewicz [Ref. 3]. Therefore in addition to 
the output created for the discretized continuous models there 
are all outputs and models available, which have been used by 
Lt P.Markiewicz [Ref. 3]. 

For the use of DISCONT FORTRAN Al it is necessary to 
specify an Input file in line 24 of DISCONT EXEC Al. This 
input file must have all necessary inputs in the same format 
as mentioned in Appendix A of Lt P.Markiewicz thesis [Ref. 3]. 
The filename and type can be chosen free as long as it matches 
the format of the CP/CMS requirements of a maximum of eight 
characters. 

For the purpose of this thesis, 42 different input 
files have been created to allow the evaluation of the 
sensitivity of the model to the following different input 
parameters 

- type of growth pattern 
~ number of failure causes and 
- number of failures per phase. 

The filenames, used in the 42 input files, express the 
setting of these different input parameters. For example, the 
filename PT1C5F3 means 

- pattern & 21 
~ number of failure causes : 5 
- number of failures/phase : 3 
The full name of the input file is then 


mrercSF3 DATA Al. 


ye 


The output for the AMSAA-D models as well as the 
results of the models chosen for comparison is written to 
GROWTH EST Al. A sample output is given in Appendix B. 

Also a control output is available. It allows us to 
track the intermediate results of single simulation runs for 
the first and last 10 replications, independent of the number 
of replications chosen. This control output is written to 
GROWTH CONTROL Al; 

The next output file is FAILURE RATE Al. It contains 
the actual reliability as well as the average of the estimated 
failure rates of each replication, which is the basis for the 
computations of the estimated reliability in the linear and 
the exponential power model. 

The last output file is MSE COMPARE Al. This file 
contains results used to evaluate the influence of changes in 
the number of failure causes and the number of failures/phase. 
It contains the average Mean Square Error, averaged over all 
phases from the distinct Mean Square Error for each phase, 


computed for the four growth models 


—)l 2... J1near modek 

- 2 .... exponential power model 

= 3ise.. MLE WEEN discoumeine 

- 4 .... exponential Regression Estimate 


In addition to these files, related to the AMSAA-D 
models, all results of Lt P.Markiewicz's simulation program 


DRG FORTRAN Al are computed and written to the same output 
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files as in their original program. The only difference is 
Eioiene fLillenames have Gm addition an "X" in front of their 
original name. In this way it is possible to run both 
simulations, DISCONT and DRG, one after the other without 
overwriting the first created output files. Therefore the file 
descriptions in Appendix A of the thesis of Lt P.Markiewicz 
are totally valid. 
3. The input data file 

These instructions should be used in connection with 
a sample input ( see Appendix B 2.). The sequence of all data 
inputs is mandatory as it is shown in the sample input file. 
The input is read from file device number 10. 

For the discretized continuous models the input can 
be the same as for the DRG FORTRAN program of Lt. 
P.Markiewicz. All steps noted in her thesis in Appendix A 3. 
are still valid. The only difference is that the first line 
of the input file should contain the filename of the input 
file to get this remark stated on top of the output file 
GROWTH EST Al. Due to this additional first line the only 
change in the steps of Appendix A 3 of Ref. 3 is that the 
mentioned line numbers must be increased by one. But all other 


input variations/possibilities stay the same. 
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4. Output file GROWTH EST Al 


The output file shows in its first line the filename 


of the input file used for the actual simulation run. The 


names of the variables are : 


= PHASE 


- ACTREL 


= NRE TS 


=e El; 


=WRE LEX 


= MSE 


= MEEWD 


="MSEWD 


= eh EGiu.S © 


= MSEREG 


self explanatory 

actual reliability 

reliability estimate, using the linear model 
Mean Square Error of the linear model 
reliability estimate using the exponential 
power model 

Mean Square Error of the exponential power 
model 

Maximum Likelihood estimate with discounting 
Mean Square Error of the MLEWD 

exponential regression estimate 


Mean Square Error of the exp.Reg.Estimate 
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APPENDIX B 


FILES AND VARIABLES 


Sample DISCONT EXEC Al file 

Sample input file : PT1C5F3 DATA Al 

Sample simulation run, using DISCONT EXEC Al, 
recorded session 

Sample output GROWTH EST Al 

Main variables used for the discrete modification 


of the continuous AMSAA model ( AMSAA-D model ) 
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1. Sample DISCONT EXEC Al file 


&TRACESOE 
KKEEKKKEKKKEKKKEKKKKEKKEKKEKKKEKEKKEKEKKREKEKKKEKEKEEKKKEKEKKEKEKKKEKKEEEK 
&FN = DISCONT 

&FN1 = GROWTH 

&TYPE THIS PROGRAM USES A DISCRETIZED CONTINUOUS MODEL 
&TYPE DO YOU NEED TO COMPILE YOUR PROGRAM ? (Y/N) 
&READ VAR &R_ COMPILE 

&IF &R COMPILE NE Y &GOTOM@=RUN 

-H FORTVS &FN 

&IF &RC EQ O &GOTO -RUN 

&TYPE Your program did not compile; check for Cerrone: 
&TYPE DO YOU WISH TO VIEW THE PROGRAM LISTING FILE? (Y/N) 
&READ VAR &RSP1l 

&IF &RSP1 EQ Y BROWSE &FN LISTING A 

&TYPE DO YOU WISH TO XEDIT THE PROGRAM FILE? (Y/N) 
&READ VAR &RESP1 

Stl Seek hor lL NBoy sex rd. 

&COMMAND XEDIT &FN FORTRAN A 

&TYPE DO YOU WISH TO RUN THE PROGRAM AGAIN? (Y/N) 
&READ VAR &RESP2 

&IF &RESP2 EQ Y &GOTO -H 

a oh Gl aaa 


-RUN 
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PPE DEE 


FILEDEF 


PELE DEF 


PELE DER 


PEE DET 


FILEDEF 


FILEDEF 


FILEDEF 


FILEDEF 


FILEDEF 


FILEDEF 


Pane DEF 


FILEDEF 


FILEDEF 


FILEDEF 


Pee DEF 


FILEDEF 


FILEDEF 


FILEDEF 


FILEDEF 


FILEDEF 


PILEDEF 


FILEDEF 


FILEDEF 


FILEDEF 


PILEDEF 


10 


82 


84 


30 


Soe 


20 


25 


81 


83 


87 


88 


90 


Bo 


40 


50 


She. 


60 


65 


70 


75 


is, 


39 


49 


16 


38 


48 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DIskK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


PTZ2C7 Fee DATA Al 

XA1 NUM Al 

XAQ9 NUM Al 

MIRE LIAB@EiS TING Al (LRECL 133 
XPRELIAB LISTING Al (LRECL 133 
XJTHESIS OUT Al 

XPTHESIS OUT Al 

XPMATRIXA LISTING (LRECL 133 
XJMATRIXA LISTING (LRECL 133 
XPREGMAT DATA Al 

XJREGMAT DATA Al 

AYSTAR LISTING (LRECL 133 
XTRIALS DATA Al 

XEST OUT Al 
XJMLEWD OUT Al 
XPMLEWD OUT Al 
XJMLESP OUT Al 
XPMLESP OUT Al 
AJREGEST OUT Al 
XPREGEST OUT Al 
XPWRES1 


OUT Al 


XPWRES2 OUT Al 


XPWRES3 OUT Al 


AJWRES1 OUT Al 


XJWRES2 OUT Al 


XJWRES3 OUT Al 
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PLE DEE 


PIPE pEe 


FILEDEF 


FILEDER 


FI LEDES 


FILE DET 


FIR EDEF 


PrP EE PE 


PLEBDEF 


FILEDED 


‘gd El ca Se oh 


FILEDEF 


a GS Och BY chy 2 


FPILEDEF 


PILEDEYr 


PEE OE 


FILE@E 


PLE DEE 


PILEDES 


PILEDEY 


FILE DER 


FIDEDEF 


PILEDE: 


Fil LEDES 


Pier, E 


mene DEF 


aye 


oval 


54 


a3 


vz 


71 


74 


S 


ay | 


76 


79 


78 


a2 


oi 


18 


i er 


94 


=e 


96 


o> 


62 


61 


64 


63 


iat 


EZ 


DISK 


Disk 


DISK 


DISK 


DISK 


DISK 


DIES 6 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


DISK 


XMLEWD1 NUM Al 
XMLEWD1 SDV Al 
XMLEWD9 NUM Al 
XMLEWD9 SDV Al 
XREG8 NUM Al 
XREG8 SDV Al 
XREG16 NUM Al 
XREG16 SDV Al 
Al 


XM1P1 NUM 


XM1P1 SDV Al 


XM2P1 NUM Al 


XM2P1 SDV Al 


AM3P1 NUM Al 


AM3SPI SDV TAL 


AM1P9 NUM Al 


XAM1P9 SDV Al 


AM2P9 Al 


AMZPO SDV ae 


XM3P9 NOM Al 


AMSPOYSDY Al 
XMLESP1 NUM Al 
XMLESP1 SDV Al 
XMLESP9 NUM Al 
AMLESP9 SDV Al 
GROWTH CONTROL Al 


FAILURE RATE Al 
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FILEDEF 13 DISK GROWTH EST Al 

FILEDEF 14 DISK ERROR EST Al 

FILEDEF 19 DISK MSE COMPARE Al 

PELEDEF 21882 SK ECY Dens iEeAl 

FILEDEF 22 DISK FRATE COMPARE Al 

FILEDEF 06 TERMINAL 

LOAD &FN (START 

&IF &RC EQ O &SKIP 9 

Tivinten:’ OlEEEDEOGghalmclia@ssnOot bun correctly; check for 
errors. 

&TYPE Do you wish to XEDIT the program file? (Y) 

&READ VAR &RESP3 

&IF &RESP3 NE Y &EXIT 2 

&COMMAND XEDIT &FN FORTRAN A 

&TYPE Do you wish to run the program again? (Y) 

&READ VAR &RESP4 

mere eREoP4 EO Y &GOTO =H 

&EXIT 2 

SEY PE LLOYD ESTP TION/ EXPECTATION®POUTPUT IS IN "LLOYD EST 
7a 

& TY PE 

eryYPE CONTROL OUTPUT IS IN THE FILE "GROWTH CONTROL Al" 

& TYPE 

&TYPE YOUR ESTIMATION OUTPUT IS IN THE FILE &FN1 EST Al 

&TYPE DO YOU WISH TO BROWSE YOUR ESTIMATION OUTPUT? (Y/N) 


&READ VAR &RESP 
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&IF &RESP EQ Y &COMMAND BROWSE &FN1 EST Al 

&TYPE PRINT YOUR OUTPUT FILE? (Y/N) 

&READ VAR &RESP7 

&IF &RESP7 EQ Y &COMMAND PRINT &FN1 EST Al 

-REDO 

&TYPE DO YOU WISH TO CHANGE THE INPUT DATA FILE (Y/N) 
&READ VAR &RESP5 

&IF &RESP5 EQ Y XEDIT &FN EXEC Al 

&TYPE DO YOU WISH TO RUN THE PROGRAM AGAIN? (Y/N) 
&READ VAR &RESP6 

&RESP56 = &CONCAT OF &RESP5 &RESP6 

&IlF GRESPS6 EO YY "GGOTO =H 

&1F &RESP6 E@=yY S&e€eroO -FuUn 

& EXIT 

Sample input file : PT1C5F3 DATA Al 

PRICES rs PATTERN 1, 5 FAILURE CAUSES; 


3 FAILURES/ PHASE 


5 NUMBER OF FAILURE CAUSES 
10 NUMBER OF PHASES ( NPHASE ) 
1 FIXED RELIABILITY OPTION 


( i: YES ; 0290) 


é} NUMBER OF FAILURES IN PHASE 1 
5 NUMBER OF FAILURES IN PHASE 2 
3 NUMBER OF FAILURES IN PHASE 3 
3 NUMBER OF FAILURES IN PHASE 4 
3 NUMBER OF FAILURES IN PHASE 5 
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ar 21) 


. 86 


~90 


~91 


e273 


~95 


a7 


oo 


~99 


wo9S8 


~84 


-oD 


mo/ 


a0 


oe 


oD 


ao 7 


wo9 


ooo 


2998 


ros 


NUMBER OF 
NUMBER OF 
NUMBER OF 
NUMBER OF 
NUMBER OF 
PROBS OF 
PROB or 
PROB OF 
PROB = OF 
PROB. OF 
PROB. OF 
PROD, -o 
PROB Or 
PROBS OFf 
PROB OF 
PROBS OF 
PROG or 
PROEGOFr 
PROB= OF 
PROB OF 
PROBS OF 
rPROBe* OF 
PROBAOF 
PROB. OF 
PROBESOF 
PROB. OF 


FAILURES IN 


FAILURES IN 


FAILURES IN 


FAILURES IN 


FAILURES IN 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 
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FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


10 


-84 


-86 


88 


ee, 


295 


~ 96 


98 


799 


oS 


oo 


84 


{35 


ory 


-o9 


oe 


94 


Ree) ay 


soo 


298 


oe 


“oo 


84 


- 86 


s62 


oon 


~94 


PROB. 


PROB« 


PROB. 


PROB. 


PROB. 


PROB. 


PROB. 


PROB. 


PROB. 


PROB. 


PROB. 


PROB. 


PROB. 


PROB. 


PROB: 


PROG. 


PROBe 


PROB. 


PROB. 


PROB. 


PROB. 


PROB: 


PROB. 


PROBs 


PROG: 


PROB: 


Oe 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OF 


OL 


Or 


OF 


OF 


OF 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 


SUCCESS 
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FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


FROM 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


CAUSE 


IN 


IN 


IN 


IN 


IN 
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IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


IN 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


PHASE 


-961 


soo 


suo) ie: 


0.0 


624712.0 


27D 


500 


PROB sO beSU ee loom RoMme CAUSE 5 IN PHASE 8 
PROB. OF SUCCESS FROM CAUSE 5 IN PHASE 9 
PROB. Of SUCCESS FROM CAUSE 5 IN PHASE 10 
NUMBER OF TRIALS AFTER FAILURE BEFORE A 
DESCOUNT IS APPSIED 


FRACTION EACH FAILURE IS DISCOUNTED 


RANDOM NUMBER SEED FOR GGUBFS UNIFORM (0,1) 


GENERATOR 


FRACTION RELIABILITY IMPROVES AFTER FAILING 


IN A PHASE 

NUMBER OF DESIRED REPETITIONS FOR THE 
SIMULATION 

INTERMEDIATE INPUT OPTION(1:INT. OUTPUT; 
O: NO INT. OUTPUT) 

SAVE ALL MLE W/ DISCOUNTING ESTIMATES 
(1: YES; 0: NO ) 

SAVE ALL MLE SINGLE PHASE ESTIMATES 
(1: YES; 0: NO ) 

SAVE ALL UNWT'D REGRESSION ESTIMATES 
(1: YES; 0: NO ) 

SAVE ALL METHOD 1 WT'D REG. ESTIMATES 
(1: YES; 0: NO ) 

SAVE ALL METHOD 2 WT'D REG. ESTIMATES 
(1: YES; 0: Ne@ } 

SAVE ALL METHOD 3 WT'D REG. ESTIMATES 


Gee vHSs sor NO > 


oll 


1 DISCOUNTING OPTION (1: STRAIGHT % ; 2: LLOYD 
METHOD) 
9 PERCENT C.I. FOR LLOYD DISCOUNTING METHOD 


( MUST HAVE A VALUE ) 


1 LLOYD DISCOUNT INTERVAL 
a ONs) WEIGHT FOR PHASE 1 
nO WEIGHT FOR PHASE 2 
203 WEIGHT FOR PHASES: 
0:3 WEIGHT FOR PHASE 2 
-03 WEIGHT FOR PHASE 1 
PbS: WEIGHT FOR PHASE 2 
25 WEIGHT FOR PHASE 1 
ay hs, WEIGHT FOR PHASE 8 
a2 WEIGHT FOR PHASES? 
2 WEIGHT FOR PHASE 10 


Sample simulation run, using DISCONT EXEC Al, recorded 
session 

BEGIN RECORDING OF TERMINAL SESSION 

Ready; T=0.01/0.04 08:49:01 

DISCONT 

THIS PROGRAM USES A DISCRETIZED CONTINUOUS MODEL 

DO YOU NEED TO COMPILE YOUR PROGRAM ? (Y/N) 

be 

VS FORTRAN COMPILER ENTERED. O84 9 2167 

+2S=$.33 


+2S=$.33 
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*XMAIN** END OF COMPILATION 1] ****xx 


VS FORTRAN COMPILER EXITED. 08:49:26 


Execution begins... 
+25=S.33 
Pe2o=o.35 
ooo. 39 
+2S=$.33 
oon oS 


EO, D ESTIMATION/ EXPECTATION OUTPUT IS IN "LLOYD EST Al" 


GONTROL OUTPUT IS IN THE FILE "GROWTH CONTROL Al" 


YOUR ESTIMATION OUTPUT IS IN THE FILE GROWTH EST Al 
DO YOU WISH TO BROWSE YOUR ESTIMATION OUTPUT? (Y/N) 
Y 

PRINT YOUR OUTPUT FILE? (Y/N) 

Y 

DO YOU WISH TO CHANGE THE INPUT DATA FILE (Y/N) 

Y 

DO YOU WISH TO RUN THE PROGRAM AGAIN? (Y/N) 

Y 

VS FORTRAN COMPILER ENTERED. 08:50:56 

+2S=$.33 


+2S=$.33 


aye) 


kKKMAIN** END OF COMPILATION 1 *#***kx 


VS FORTRAN COMPILER EXITED. O86: 51208 


Execution begins... 
t2S—cees 
2 o—o. 05 
+2S=$.33 
2 o— oS 
feos. o0 


LLOYD ESTIMATION/EXPECTATION OUTPUT IS IN "LLOYD EST Sas 


CONTROL OUTPUT IS IN THE FILE” “GROWTH CONTRCGE 


YOUR ESTIMATION OUTPUT IS IN THE FILE GROWTH EST Al 
DO YOU WISH TO BROWSE YOUR ESTIMATION OUTPUT? (Y/N) 
Y 

PRINT YOUR OUTPUT FILE? (Y/N) 

N 

DO YOU WISH TO CHANGE THE INPUT DATA FILE (Y/N) 

N 

DO YOU WISH TO RUN THE PROGRAM AGAIN? (Y/N) 

N 

Ready; T=27.06/30.92 08:52:03 


EXEC REC Orr 
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4. 


Sample output GROWTH EST Al 


SIMULATION INPUT FILE : PTIC5F3 


PHASE ACTREL LINREL MSEL RELEXP MSEE MLEWD MSEWD REGEST MSEREG 


~ 39842 
-42811 
48079 
29924 
760995 
. 10268 
mol 2 
we 9gee 
Bes 2105), 
99004 


OW worn Of W PO kK 


— 


iowa wom soo see OI7> s3S78O .0517 .36841 .0553 
-40747 .0404 .56384 .0304 .34822 .0547 .37926 .0569 
-45315 .0418 .59035 .0250 .39846 .0564 .43754 .0432 
ep 145 syO 4237 eeo lee 2 i ie 46259 .0561 .50047 .0374 
Pome Ome d¢eeGoee oO? 7 sol9TS .0585 .56433 .0296 
~ Ooo Uomene cs IO. 1 1CUAmeOdezeots36 .0402 .64388 .0213 
meee .0228 2f 9529 .UT0S .7/3483 .0289 .73642 .0147 
-88128 .0071 .89087 .0045 .86940 .0091 .84127 .0069 
poogmemeoesse. 93576 20027 292529 .0054 .90832 .0030 
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Main variables used for the discrete modification of the 


continuous AMSAA model ( AMSAA-D model ) 


TRIOT 
NFAPH (K) 
NFAPHK 
FRATE 


FRHAT (K) 


FSHAT (K) 


XAHAT, XBHAT 


2** 
AREL(K) 
RHAT (K) 
bai 


YI 


number of trials in a phase 

number of failures in phase k 

number of failures in a phase 

failure rate estimate of the AMSAA-D model 
failure rate estimate for phase k using the 
AMSAA-D model 

minimum variance unbiased failure rate 
estimate for phase k 

regression estimates a, b in one phase and 
repetition 

auxiliary variables 

actual reliability in phase k 

failure rate estimate at phase k 


ln of the total number of trials up to phase k 


ln of the failure rate estimate at phase k 


ays, 


LSSUM(4,K) 


ANUMER 


ADENOM 


ESTRN (K) 


ESTRNE (K) 


RNHAT (K) 


RNHATE (K) 


AVFR(K) 


AVRN (K) 


MSE(K) 


AVRNE(K) 


MSEE (K) 


MLEWD (3, K) 


MSEWD(K) 


summands of the least square estimate a in 
phase k 

numerator to compute a 

denominator to compute a 

failure rate estimate in phase k using the 
linear model 

failure rate estimate in phase k using the 
exponential power model 

Summation of the reliability estimate using 
the linear model 

Summation of the reliability estimate using 
the exponential power model. 

average failure rate estimate for phase k 
average reliability estimate for phase k, 
using the linear model 

Mean Square Error of the linear model 
reliability estimation 
average reliability estimate for phase k, 
using the exponential power model 

Mean Square Error of the exponential power 
model reliability estimation 

average reliability using the maximum 
likelihood estimation with discounting 


Mean Square Error of the MLEwd 
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REGEST (3, K) 


MSEREG (K) 


MSEERR (I,K) 


l= 
l= 


AVGMSE(T) 


average reliability using the exponential 
regression estimate 

Mean Square Error of the exp.Regr.Estimate 
average Mean Square Error over all phases k 
for the i methods. 

1 : AMSAA-D, linear model 

2 : AMSAA-D, exponential power model 

3 : MLEwd 

4 : exp.Regr.Estimate 

average Mean Square Error over all phases for 


method i 
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APPENDIX C 


FAILURE RATE FOR THE GEOMETRIC AND PASCAL DISTRIBUTION 


The failure rate for a discrete probability distribution 
is defined as 
P(N=n] 


‘n ” SeNSny 2° a ( C. i 


In this treatment the random variable N is the number iia 


ee failure; that is, N has a 


trials up to-and including the = 
Pascal distribueron. 
1. Case 1: The number of failures equals 1. 
In this case the number of Bernoulli trials, N, 


required to obtain the first failure, has a geometric 


distribution with parameter p. Thus 


PIN =n) = p"-l(i-p) , n=1,2,... ( C.2 ) 


It follows that 


PEN >see 
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Thepetone trom equatton .( ©€.1),the failure rate 


fanNCt 10Nn fn LS 


pee (1 =p) 
pn-i 
or 
2 oe ( (Gc) 


The parameter p of the geometric distribution represents the 
reliability of the item under test and therefore it follows 


that 


Rey = 1 - ftp ( (C4) 


This relation suggests the use of equation (II.6) to convert 


the estimated failure rate into a reliability value; i.e. 
bel Ss Il =a (Ga 9) 


Even when the number of failures ina phase is r > 1, 
we are still interested in modeling the growth of the 
probability of success on a single trial. Consequently 
equation C.5 provides the correct expression for an estimate 


of reliability in terms of the failure rate estimate. 
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2. Case 2 : The number of failures equals r. 
In this case N has a Pascal distribution with 
parameters p and r. 


PIN =n] = (R73)p™ F(a - p)* 


and 


P[N > nN] = P(N > nal )e=] el. se 


where Y is binomial with parameters 1-p and n-1. Therefore 


r-l : ; 
PIN-> nj) = >. (9o+)p? ss See 
j=o 7 
and 
n-l n-r 1 
(DTT) p™ * (1-p) 
BF ( Cc. 6m 
Yr) r=t : ; 
EMSA ypn’ J (a-p)J 
j=0 


It is important to realize that even though C.6 is the 
failure rate function when testing until r failures occur, it 
is not this failure rate that we are modeling in the AMSAA-D 
model. We are modeling the growth of p only; i.e. the growth 
of the probability of success ( reliability ) on a single 


se aad Ws Waa 
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APPENDIX D 


FIGURES 16 TO 45 
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RELIABILITY GROWTH ESTIMATION 
AMSAA model, pattern 2 
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Figure 16 : Reliability growth estimation with the discrete 


AMSAA model, pattern 2 
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RELIABILITY GROWTH ESTIMATION 
AMSAA model, pattern 4 
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Figure 17 : Reliability growth estimation with the discrete 
AMSAA model, pattern 4 
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RELIABILITY GROWTH ESTIMATION 
Pattern 1, ceuses = 3, failura/phasa = 3 
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Figure 18 : Reliability growth estimation and Mean Square 
Error, pattern 1, 3 failure causes, 3 failures/phase 
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RELIABILITY GROWTH ESTIMATION 
Pattern 2, causes = 3, failure/phase 
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Figure 19 


Reliability growth estimation and Mean Square 
rEror, 


pattern 2, 3 failure causes, 3 failures/phase 
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RELIABILITY GROWTH ESTIMATION 
Pattern 3, cauesse = 3, falilure/phese * 3 
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Frror, pattern 3, 3 failure causes, 3 failures/phase 


66 


RELIABILITY GROWTH BSTIMATION 
Pattern 4, caveee * 3, 
Fe er ae 


failure/pheee = 3 
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RELIABILITY GROWTH ESTIMATION 
Pettern 6, causeee 8 3, fallure/phese = 3 
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Figure 22 : Reliability growth estimation and Mean Square 


Frror, pattern 5, 3 failure causes, 3 failures/phase 
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RELIABILITY GROWTH BSTINATION 
Pattern 6, caveee = 3, feiluresphase * 3 
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RELIABILITY GROWTH ESTIMATION 
Pattern 1, causes = 6, failure/phaee = 1 
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Figure 24 : Reliability growth estimation and Mean Square 
Error, pattern 1, 5 failure causes, 1 failure/phase 
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Figure 26 : Reliability growth estimation and Mean Square 
Error,” pattern 3, 5 failure causes, 7 tame ones 
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Figure 27 : Reliability growth estimation and Mean Square 
Mmieror, pattern 4, 5 failure causes,,1 failure/phase 
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Figure 28 : Reliability growth estimation and Mean Square 
Error, pattern 5, 5 failure causes, 1 failure/phase 
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Figure 30 : Reliability growth estimation and Mean Square 
Error, pattern 1, 7 failure causes, 3 failures/phase 
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Figure 31 : Reliability growth estimation and Mean Square 
Error, pattern 2, 7 failure causes, 3 failures/phase 
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Figure 32 : Reliability growth estimation and Mean Square 
Error, pattern 3, 7 failure causes, 3 failures/phase 
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Figure 33 : Reliability growth estimation and Mean Square 
Error, pattern 4, 7 failure causes, 3 failures/phase 
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Figure 34 : Reliability growth estimation and Mean Square 
Error, pattern 5, 7 failureveaueoc me failures/phase 
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Figure 35 : Reliability growth estimation and Mean Square 
Error, pattern 6, 7 failure causes, 3 failures/phase 
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Figure 36 : Mean Square Error for both AMSAA-D models as a 
function of phases and number of failure causes, pattern 2, 
failures/phase = 3 
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Figure 37 : Mean Square Error for both AMSAA-D models as a 
function of phases and number of failure causes, pattern 3, 
failures/phase = 3 
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Figure 38 : Mean Square Error for both AMSAA-D models as a 
function of phases and number of failure causes, pattern 4, 
failures/phase = 3 
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Figure 40 : Mean Square Error for both AMSAA-D models as a 
function of phases and number of failure causes, pattern 6, 
failures/phase = 3 
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Figure 41 : Mean Square Error for both AMSAA-D models as a 
function of phases and failures/phase, pattern 2, failure 
Causes = 5 
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Figure 42 : Mean Square Error for both AMSAA-D models as a 
function of phases and failures/phase, pattern 3, failure 
causes = 5 
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Figure 43 : Mean Square Error for both AMSAA-D models as a 
function of phases and failures/phase, pattern 4, failure 
causes = 5 
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Figure 45 : Mean Square Error for both AMSAA-D models as a 
function of phases and failures/phase, pattern 6, failure 
causes = 5 


Salk 


APPENDIX E 


GLOSSARY OF THE DIFFERENT RELIABILITY GROWTH MODELS 


1. Maximum Likelihood Estimate Model 
2. The Exponential Regression Model 
3. The Discrete AMSAA Model 

4. The AMSAA-D Model 

a. Linear AMSAA-D Model 


b. Exponential Power AMSAA-D Model 


22 


1. Maximum Likelihood Estimate Model 
The reliability estimate R,, of the true reliability R is 


computed as 
CGecateetrials — total failures 


Ee | totale emia ls 


i> 
| 


If failure discounting is applied then the reliability 


estimate Re is evaluated as 


total trials - total adjusted failures 


Eker teal trials 


> 


The estimation is performed after each phase and, if 
failure discounting is applied, the actual test data are 
updated with the use of test data from previous phases. This 
model is denoted as MLEwd. 

2. The Exponential Regression Model 

The exponential regression model obtains sequentially 

updated estimates Bh, of the true reliability R after the 


tie 
eh 


Ji phase. The basic model for Ry is 


The parameter A, = a+ Bk is estimated by A, where 
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A A 
The estimates a, and By. are least square regression estimates 


k 
for a and $8: Equations Wie ay and B,. are developed in 


Reference 4. 
The exponential regression estimate Raw of the true 


reliability R is then finally 


el < 


emote he Py) | ae 


This model is denoted as exp.Regr.Estimate. 
3. The Discrete AMSAA Model 


In the Discrete AMSAA model, the equation for the 


En 


probability of success, R on each trial in the i testing 


je? 


phase is given by 


Z - »7P _ ym 
(1 = RN = AT ee ee 
where 
Ny number of tests in test phase i 
k 
T, = 2 N 
k 4=1 7 


After each new test phase is completed the maximum likelihood 
estimates for \ and 8 are recomputed to provide the MLE for 
R, - The two equations which must be solved after testing is 


completed in phase k are as follows [Ref. 6]: 
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= — eee ee AT? In T; = al ae In Ty, 1 =) 
= AT? — — . IN; San ers AT? | 
and, 

k 

ee ee [TP —T?_,| =0 


= | fare-ate,]  [Ne-at?+aTe, 


Numerical methods and a computer are needed to solve these 
equations. 
4. The AMSAA-D Model 

The Continuous AMSAA Model expresses the instantaneous 
failure rate fp after TT time units of testing by 


a k 


-a 


f = JS, See er 


TT). 


where Jes is the total accumulated test time over k phases. 
This model is also known as the Duane Model. After examining 
a large variety of data, Duane noticed that the plot of the 
cumulative failure rate Ap versus cumulative test time on log- 
log paper produced a straight line. From this he inferred a 
linear relationship between log An and log T. Consequently, 
he wrote the following model for cumulative failure rate 

—a 


Ap = kT 


=) 





where yale — 


N(T) : total failures Up @eemeremn 


k and a are unknown parameters. 


Thus 
Ney es Kr 
Al 
N(T) = kr?+74 


The instantaneous failure rate fo is the change in N(T) per 


unit time. Thus 


_ GN(T) _ _ =a 
fo aT (ell a) kT 


The AMSAA model departs at this point from Duane by using 
maximum likelihood methods to estimate the unknown parameters. 
Letting a = 1-8 we have 

aa Bet 
which is the Weibull failure rate function. Consequently AMSAA 
develops maximum likelihood estimates for $B and k assuming 
time to failure within a phase has a Weibull distribution. 
This assumption makes the accuracy of their model succeptible 
to variations in the underlying failure distribution. Woods 
[Ref. 4] uses regression methods to estimate the parameters 
and does not assume an underlying distribution. This makes his 


procedure more robust. Similar procedures are followed in the 


AMSAA-D model. 
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The AMSAA-D model relates the failure rate fy, and the end 
of k test phases to the total number of failures Ny} across 


the first k test phases by 
-a 
Lip mo a) No 


a. Linear AMSAA-D Model 


In this model the reliability estimate Raa of the true 


melriability R at the end of phase k is computed as 


Gk 


> 
lI 
hr 
j 
Hh > 


1G Eke 


b. The exponential Power AMSAA-D Model 


In this model the reliability estimate Re of the true 


reliability Rip at the end of phase k is computed as 


—f 
& = tk 
Ri, =e 


oy, 
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